-
Notifications
You must be signed in to change notification settings - Fork 21.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix possible padding length overflow in DistributedSampler #45329
Conversation
Codecov Report
@@ Coverage Diff @@
## master #45329 +/- ##
==========================================
- Coverage 68.08% 68.08% -0.01%
==========================================
Files 393 393
Lines 50960 50963 +3
==========================================
+ Hits 34698 34699 +1
- Misses 16262 16264 +2
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the fix! Could we add a unitttest for this to ensure no future regression? The file distributed_test.py
has a DistributedSampler test which you can use as an example.
💊 CI failures summary and remediationsAs of commit 2d622d2 (more details on the Dr. CI page):
🕵️ 3 new failures recognized by patternsThe following CI failures do not appear to be due to upstream breakages: pytorch_macos_10_13_py3_test (1/3)Step: "Test" (full log | diagnosis details | 🔁 rerun)
|
Sure 👍 I added the unit test assuring that tiny datasets are adequately padded in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rohan-varma has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
@rohan-varma merged this pull request in a699108. |
@rohan-varma merged this pull request in a699108. |
Fixes #45324
This fix handles cases for
len(dataset) * 2 < num_replica
in DistributedSampler. (which previous code resulted in error.)